Skip to content

fix(storage): sign uploads for server-allocated object keys - #2014

Open
Zetazzz wants to merge 2 commits into
mainfrom
fix/upload-object-identity-pr
Open

Zetazzz wants to merge 2 commits into
mainfrom
fix/upload-object-identity-pr

Conversation

@Zetazzz

@Zetazzz Zetazzz commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

A pending upload previously reused a caller-selected object location, allowing unconfirmed content to overwrite a confirmed original or a derived image.

Use the storage recorder's server-allocated physical key for every PUT, projection and response. Dedup only visible confirmed rows by logical_key, bucket and content hash; pending attempts get separate objects. Reject missing/empty/echoed recorder identities before signing. Preserve checksum and upload-id bindings. File-reference URLs now validate derived// children against the readable source row. Update all three storage fixture recorders and HTTP regressions to the same contract.

Coordinated deployment: this is the constructive half of the constructive-db generator/worker/GC change. The production recorder must return id + independent key and files must expose logical_key. Per constructive-db/DEVELOPMENT.md, publish this package first; one subsequent DB PR will combine the generator, regenerated SQL and published dependency bump. No old-key fallback or bucket-policy change is introduced. Existing logical-key replacement authorization remains unchanged.

Validation:

  • 112 plugin tests (11 suites) pass against local RustFS.
  • 90 HTTP integration tests (7 suites) pass: uploads, pending retries, custom logical keys, tenant selectors/routing, projection columns, live download gating and derived URLs/RLS.
  • Selected dependency build and changed-file lint pass.
  • Three attacker-selected hash/original/derived key regressions fail against pre-fix plugin source and pass after the fix.
  • Joint local DB verification: 19 real-login cases cover same-user/same-organization members, failed confirmation, image processing/download and GC; wider storage/worker suites also exercised. Final release-version verification belongs to the DB PR.

@luxor-code-review

luxor-code-review Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Review complete. 🟠 2 high · 🟡 1 medium

💬 Inline comments (3)

  • 🟠 Dedup query references logical_key no in-repo schema defines — plugin.ts:578
  • 🟠 Recorder result unvalidated: undefined or caller-controlled S3 key — storage-file-recorder.ts:67
  • 🟡 Re-upload at a custom logical key unique-violates on fixture recorders — plugin.ts:597

The change reworks upload deduplication and the storage recorder contract in graphile-presigned-url-plugin. Uploads now probe for an existing row keyed by bucket, logical key, and content hash; a match returns the existing object instead of re-uploading, and the old DELETE-then-retry path is gone. The recorder's app_files_record_file call must now return both id and the server-allocated key, which is used to sign the presigned PUT.

Files Change
src/plugin.ts Dedup probe query, reuse of an existing object's physical key on match, removal of the re-upload DELETE path
src/storage-file-recorder.ts, src/types.ts Recorder now returns id + server-allocated key; types updated
__tests__/*.test.ts Mocked probe/recorder responses updated to the new contract; dedup-liveness and attacker-key test cases added
README.md Documents the new dedup and recorder-contract behavior

Three issues survived verification: the dedup query references a logical_key column no in-repo schema defines, the recorder result is cast without validating the key field, and a re-upload at a custom logical key can unique-violate with recorder versions that don't honor the logical key.

Reviewed commit: 2a76dcb

@luxor-code-review luxor-code-review Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR reworks the presigned-URL upload plugin to deduplicate re-uploads against prior uploaded/processed rows and to require the storage recorder to return a server-allocated S3 key, removing the old delete-and-retry path.

Key findings

  • 🟠 Dedup query references logical_key no in-repo schema defines — plugin.ts:578
  • 🟠 Recorder result unvalidated: undefined or caller-controlled S3 key — storage-file-recorder.ts:67
  • 🟡 Re-upload at a custom logical key unique-violates on fixture recorders — plugin.ts:597

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant